Up to date standardized video coding methods are based on hybrid coding. Hybrid coding provides a coding step in the time domain and a coding step in the spatial domain. First, the temporal redundancy of video signals is reduced by using a block based motion compensated prediction between the image block to be coded and a reference block from an image that has already been transmitted determined by a motion vector. The remaining prediction error samples are arranged in blocks and are transformed into the frequency domain resulting in a block of coefficients. These coefficients are quantized and scanned according to a fixed and well-known zigzag scanning scheme, which starts with the coefficient representing the DC value. According to a typical representation, this coefficient is positioned among the low frequency coefficients in the top left corner of a block. The zigzag scanning produces a one-dimensional array of coefficients, which are entropy-coded by a subsequent coder. The coder is optimised for an array of coefficients with decreasing energy. Since the order of coefficients within a block is predetermined and fixed, the zigzag scanning produces an array of coefficients of decreasing energy, if the prediction error samples are correlated. The subsequent coding step may then be optimised for such a situation. For this purpose, the latest standard H.264/AVC proposes Context-Based Adaptive Binary Arithmetic Coding (CABAC) or Context-Adaptive Variable-Length Coding (CAVLC). However, the coding efficiency of the transform is only high, if the prediction error samples are correlated. For samples being only marginally correlated in the spatial domain, the transform is less efficient.
The spatial redundancy may be reduced by blockwise transform coding of the resulting prediction error. For the purpose of transform coding, H.264/AVC applies an integer transform for coding a macroblock of 16×16 picture elements, which is similar to the Discrete Cosine Transform. The size of the transform can be changed for each macroblock between 8×8 or 4×4 picture elements, signaled by side information. In the first case, 4 8×8 transforms and in the second case, 16 4×4 transforms are applied for the macroblock. Dependent on the size of the applied transform different quantizations procedures are performed. Most of the encoding strategies that are applied in the official reference software are described in K.-P. Lim, G. Sullivan, T. Wiegand, “Text description of Joint Model Reference Encoding Methods and Decoding Concealment Methods”, Joint Video Team (JVT), doc. JVT-K049, Munich, Germany, March 2004.
In the case of an 8×8 transform the quantization is performed in the official reference software as follows. For each of the four 8×8 prediction error blocks Bj (j=0, . . . , 3) of a macroblock the transform is performed resulting in a block of 8×8 coefficients ck,j (k=0, . . . , 63). Each coefficient is quantized by a scalar quantizer as shown in FIG. 1. The quantized coefficients c′k,j (k=0, . . . , 63, j=0, . . . , 3) are scanned by the well known Zigzag scan starting at the DC-coefficient resulting in a one dimensional array of 64 quantized coefficients c′k,j.
Subsequent to these coding steps, a second quantization step is performed in order to prevent that single quantized coefficients unequal to zero in an 8×8 block are coded. The coding of these single quantized coefficients unequal to zero may require a high data rate and may reduce the distortion only marginally. For this purpose, a value Ik,j, characterizing the importance of the quantized coefficient, is associated to each of the 64 quantized coefficients. Three cases are distinguished. If a quantized coefficient has an absolute value of one, the value Ik,j is dependent on the number Nk,j of preceding zero coefficients. The dependency between Ik,j and Nk,j is shown in the following table 1:
TABLE 1Nk,j0123456789101112131415Ik,j3333222222221111Nk,j16171819202122232425262728293031Ik,j1111111100000000
Accordingly, if a quantized coefficient has an absolute value of one and there are 3 or less preceding zero coefficients, the value characterizing the importance of the corresponding quantized coefficient is set to 3. If there are 24 or more preceding zero coefficients the value characterizing the importance of the corresponding coefficient is set to 0.
For each of the 64 quantized coefficients c′k,j, which has an absolute value larger than one, Ik,j is set to a very large value such as 999999. For each of the 64 quantized coefficients c′k,j, which is zero, Ik,j is set to zero. All 64 values Ik,j are added resulting in the sum
      I    j    =                    ∑        63                    k        =        0              ⁢                  I                  k          ,          j                    .      In the case that the sum Ij is smaller than the threshold 5, all quantized coefficients of the 8×8 block are set to zero and consequently Ij is also set to zero.
After the determination of the values I1, I2, I3, and I4 of the four 8×8 prediction error blocks B1, B2, B3, and B4 these four values are added resulting in the sum IMB for the whole Macroblock. In the case that IMB is smaller than the threshold 6, all 256 quantized coefficients of the Macroblock are set to zero.
In case of an 4×4 transform the quantization is performed as follows. Each 8×8 predication error block Bj (j=0, . . . , 3) of a Macroblock is divided into four 4×4 blocks Pj,i (j=0, . . . , 3, i=0, . . . , 3). For each of the four 4×4 blocks, the transform is performed resulting in a block of 4×4 coefficients ck,j,i (k=0, . . . , 15, j=0, . . . , 3, i=0, . . . 3). Each coefficient is quantized by a scalar quantizer as shown in FIG. 1. The quantized coefficients c′k,j,i of each of the four 4×4 blocks are zigzag scanned starting at the DC-coefficient resulting in a one dimensional array of 16 quantized coefficients c′k,j,i.
Subsequent to these coding steps, a second quantization step is performed in order to prevent that single quantized coefficients unequal to zero in an 8×8 block are coded. In this second quantization step, all 64 quantized coefficients of the four 4×4 blocks of the 8×8 block are taken into account. For the quantization purpose, a value Ik.j,i is associated to each of the 64 coefficients. Three cases are distinguished. If a quantized coefficient has an absolute value of one, the value Ik,j,i is dependent on the number Nk,j,i of preceding zero coefficients. The dependency between Ik,j,i and Nk,j,i is shown in the following table 2:
TABLE 2Nk,j,i0123456789101112131415Ik,j,i3221110000000000
For each of the 64 quantized coefficients c′k,j,i, which has an absolute value larger than one, Ik,j,i is set to a very large value such as 999999. For each of the 64 quantized coefficients c′k,j,i, which is zero, Ik,j,i is set to zero. All 64 values Ik,j,i are added for each 8×8 prediction error block Bj resulting in the sum
      I    j    =                    ∑        3                    i        =        0              ⁢                            ∑          15                          k          =          0                    ⁢                        I                      k            ,            j            ,            i                          .            In the case that the sum Ij is smaller than the threshold 5, all quantized coefficients of the 8×8 block are set to zero and consequently Ij is also set to zero.
After the determination of the values I1, I2, I3 and I4 of the four 8×8 prediction error blocks B1, B2, B3, and B4, these four values are added resulting in the sum IMB for the whole macroblock. In the case that IMB is smaller than the threshold 6, all 256 quantized coefficients of the macroblock are set to zero.
Accordingly, in the case of an 8×8 transform as well as in the case of an 4×4 transform coefficients of prediction error blocks are quantized and a further quantization step is performed in order to prevent that single quantized coefficients unequal to zero in a 8×8 block are coded. Therefore, rules are given for setting some coefficients to zero, which are considered to be of minor relevance, in order to significantly reduce the data rate while at the same time the distortion would only be increased marginally. However, this known procedure is unsatisfactory.